Convergence Rate of Frank-Wolfe for Non-Convex Objectives
نویسنده
چکیده
We give a simple proof that the Frank-Wolfe algorithm obtains a stationary point at a rate of O(1/ √ t) on non-convex objectives with a Lipschitz continuous gradient. Our analysis is affine invariant and is the first, to the best of our knowledge, giving a similar rate to what was already proven for projected gradient methods (though on slightly different measures of stationarity).
منابع مشابه
Frank-Wolfe methods for geodesically convex optimization with application to the matrix geometric mean
We consider optimization of geodesically convex objectives over geodesically convex subsets of the manifold of positive definite matrices. In particular, for this task we develop Euclidean and Riemannian Frank-Wolfe (FW) algorithms. For both settings we analyze non-asymptotic convergence rates to global optimality. To our knowledge, these are the first results on Riemannian FW and its convergen...
متن کاملGreedy Algorithms for Cone Constrained Optimization with Convergence Guarantees
Greedy optimization methods such as Matching Pursuit (MP) and Frank-Wolfe (FW) algorithms regained popularity in recent years due to their simplicity, effectiveness and theoretical guarantees. MP and FW address optimization over the linear span and the convex hull of a set of atoms, respectively. In this paper, we consider the intermediate case of optimization over the convex cone, parametrized...
متن کاملLinear Convergence of a Frank-Wolfe Type Algorithm over Trace-Norm Balls
We propose a rank-k variant of the classical Frank-Wolfe algorithm to solve convex optimization over a trace-norm ball. Our algorithm replaces the top singular-vector computation (1-SVD) in Frank-Wolfe with a top-k singular-vector computation (k-SVD), which can be done by repeatedly applying 1-SVD k times. Our algorithm has a linear convergence rate when the objective function is smooth and str...
متن کاملOn the Global Linear Convergence of Frank-Wolfe Optimization Variants
The Frank-Wolfe (FW) optimization algorithm has lately re-gained popularity thanks in particular to its ability to nicely handle the structured constraints appearing in machine learning applications. However, its convergence rate is known to be slow (sublinear) when the solution lies at the boundary. A simple lessknown fix is to add the possibility to take ‘away steps’ during optimization, an o...
متن کاملDecentralized Frank-Wolfe Algorithm for Convex and Nonconvex Problems
Decentralized optimization algorithms have received much attention due to the recent advances in network information processing. However, conventional decentralized algorithms based on projected gradient descent are incapable of handling high dimensional constrained problems, as the projection step becomes computationally prohibitive to compute. To address this problem, this paper adopts a proj...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- CoRR
دوره abs/1607.00345 شماره
صفحات -
تاریخ انتشار 2016